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ABSTRACT 

This  paper  shows  that  it  is  possible  for  an  adaptive  transversal 
prediction  filter  to  outperform  the  fixed  Wiener  predictor  of  the 
same  length  for  narrowband  input  signal  embedded  in  Added 
White  Gaussian  Noise  (AWGN).  The  error  transfer  function 
approach,  which  takes  into  account  of  the  correlation  of  predictor 
error  feedback  and  input  signal,  is  derived  for  stationary  and 
chirped  input  signals.  It  shows  that  with  a  narrowband  input 
signal,  the  nonlinear  effect  is  small  for  a  1-step  predictor,  but 
increases  in  magnitude  as  the  prediction  distance  is  increased. 
We  also  show  that  the  LMS  predictor  uses  information  from  the 
past  errors  more  effectively  than  the  Recursive  Least  Square 
(RLS)  predictor,  as  a  consequence,  the  magnitude  of  nonlinear 
effects  of  the  LMS  predictor  are  more  significant  than  for  the 
RLS  predictor. 


1.  INTRODUCTION 

The  least-mean-square  (LMS)  adaptive  filter  is  widely  used  in 
many  applications  partly  due  to  the  simplicity  of  its 
implementations  [1],  The  simplicity  belies  the  fact  that  the 
adaptive  LMS  filter  is  a  complex  nonlinear  estimator  [2]. 
Traditional  analysis  of  adaptive  filter  performance  typically 
invokes  a  set  of  assumptions  that  the  filter  error  output  is 
independent  of  the  current  input  data  [1].  It  has  been  shown  that 
these  well  known  assumptions  mask  the  nonlinear  effects  that 
arise  in  LMS  adaptive  filters.  It  is  has  been  shown  that  it  is 
possible  for  LMS  algorithms  to  outperform  finite-length  Wiener 
filter  for  the  case  of  adaptive  channel  equalization  for  sinusoidal 
and  AR1  interference  suppression  [3]  and  for  adaptive  noise 
cancellation  for  narrowband  AR1  signal  when  the  primary  and 
reference  signals  have  slightly  different  frequencies  [2].  An  error 
transfer  function  approach  is  also  derived  in  [3]  to  compute  the 
total  MSE  of  LMS  channel  equalizer. 

In  this  paper,  the  nonlinear  effects  for  a  third  application  of 
adaptive  filters,  adaptive  prediction,  is  studied.  The  class  of  input 
signals  which  will  be  considered  for  adaptive  prediction  are  the 
stationary  and  chirped  narrowband  input  signals  for  varying  chirp 
rates  and  bandwidth.  This  class  of  signals  has  been  used  to 
represent  a  signal  whose  spectrum  is  frequency  offset  and  shifted 
with  time  in  a  nonstationary  mobile  communications 
environment  [4].  This  class  of  signals  are  different  from  those 
considered  in  [2]  [3]  because  they  have  a  time-varying  Power 
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Spectral  Density  (PSD).  Since  they  do  not  have  a  fixed  PSD,  the 
error  transfer  function  approach  is  not  directly  applicable. 
However,  since  the  chirped  signal  has  a  constant  spectral  shifting 
rate,  this  special  class  of  nonstationary  inputs  can  be  analyzed  as 
stationary  inputs  in  the  chirp  transform  domain.  It  is  proven  in 
this  paper  that  the  MSE  of  chirped  input  signal  using  the  standard 
LMS  predictor  equals  the  MSE  of  the  stationary  input  signal 
using  a  transformed  LMS  predictor.  An  error  transfer  function 
approach  is  derived  for  the  transformed  LMS  algorithm  with 
stationary  input  signals  to  calculate  the  MSE  of  chirped  signal 
prediction.  To  bound  the  performance  of  the  adaptive  LMS 
predictor,  the  MSE  of  the  optimal  estimator  (which  is  the  infinite 
length  1-step  causal  Wiener  predictor)  is  calculated. 

To  compare  the  magnitude  of  nonlinear  effects  of  the  LMS  and 
RLS  predictors,  the  error  feedback  transfer  function  is  also 
derived  for  the  RLS  algorithm.  By  comparing  the  contributions 
of  past  errors  to  current  estimations  of  the  two  algorithms,  it  is 
shown  that  LMS  uses  more  information  from  past  prediction 
errors  than  RLS,  consequently  the  nonlinear  effects  can  be 
significant  for  LMS  algorithm,  but  only  barely  observable  for 
RLS  algorithm. 

2.  BACKGROUND 

It  is  shown  in  [4]  that  first  order  autoregressive  (AR1)  process 
provides  a  reasonable  approximation  to  a  BPSK  input  signal.  The 
AR1  process  has  the  recursive  equation 

Sn=™n- l+V„  (1) 

where  {  Vn  }  is  a  white  noise  process,  with  a]  =  Ps  (1  —  a1 )  . 

It  is  also  shown  in  [4]  that  the  corresponding  chirped  AR1  signal 
has  the  following  form 

_\ 

sn=anV  2sn_ !  +  v*  (2) 

where  Q  =  eJ(0  shifts  the  center  frequency  of  the  spectrum  and 

W  =eJ¥  linearly  shifts  the  center  frequency  with  time.  At  the 
receiver  the  signal  is  given  by 

x„=s„+n„,  (3) 

where  \nn }  is  a  white  noise  process  with  power  Pn  . 

Figure  1  represents  a  linear  predictor  structure  to  be  analyzed, 
where  W (n)  is  the  adaptive  filter.  The  weight  update  equation 
for  l.MS  algorithm  is 
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W(n  +  \)  =  W(n)  +  nX"(n)e„,  (4) 

where  JJ.  is  the  step-size  parameter  for  LMS  algorithm  and  *  is 
the  complex  conjugate.  The  error  update  equation  is  given  by 

ent,=x„t,-WT(n  +  l)X(n+\)  (5) 

The  fixed  Wiener  predictor  weight  and  corresponding  MSE  are 
given  as 

Wa  =  R-'P ,  (6) 

Jw  =  Ps  +  Pn-P"W0,  (7) 

where  R  is  the  autocorrelation  matrix  of  the  input  signal  vector, 
P  is  the  cross-correlation  of  the  input  signal  vector  with  the 
desired  response,  Ps  is  the  power  of  input  signals,  and 

Jw  =  £[|  dn  -  Wl  X (a)  |2  ]  is  the  MSE  of  finite  Wiener  filter.  It 

has  been  shown  in  [4]  that  the  MSEs  of  Wiener  filters  are  the 
same  for  stationary  and  chirped  input  signals. 

3.  THE  LMS  PREDICTOR  FOR  CHIRPED  INPUT 
SIGNALS 

The  error  transfer  function  approach  derived  in  [3]  requires  input 
signals  to  be  wide-sense  stationary,  i.e.,  the  input  signals  have  a 
fixed  PSD.  For  chirped  input  signals,  the  PSD  is  constantly 
shilling  with  time,  this  approach  is  not  directly  applicable. 
However,  the  adaptive  recovery  of  narrowband  chirped  signal 
using  A  —  step  predictor  has  one  important  characteristic,  i.e., 
the  frequency  offset  among  the  input  signal  taps  is  the  chirp  rate 
y/  ,  and  the  frequency  offset  between  the  desired  response  xcn 
and  input  signal  vector  Xc (n)  is  Ay/  .  By  multiplying  the 
chirped  input  signal  by  a  negative  frequency  offset,  we  can 
transform  the  signal  component  scn  to  its  stationary  form  s„  and 
leave  the  noise  component  nn  unchanged  since  the  AGWN  has  a 
constant  spectral  envelope  across  all  frequency.  In  the  following, 
it  is  shown  that  the  above  transform  will  not  change  the  MSE  of 
chirped  signal  predictor,  consequently  the  error  transfer  function 
approach  is  applied  to  the  transformed  signals.  It  provides  the 
MSE  of  I  MS  predictor  for  the  chirped  input  signals. 

3.1  Equivalence  of  MSEs 

For  a  chirped  input  signal  xc(n)  =  sc (n)  +  n(n),  n  =  1,2,... , 
where  sc(n)  has  frequency  Q  =  e><u°  and  chirp  rate  ¥  =  e*  , 

\vc  define  an  unchirped  process,  x(n)  =  Cl'nHt  2xc(n ), 
/i  =  0,1,....  It  will  be  shown  next  that  this  operation  will 
transform  that  chirped  input  signal  to  a  stationary  baseband 
signal.  The  LMS  algorithm  for  chirped  input  signal  xc(n)  is 
given  by 

Wc (n+  1)  =  Wc{n)  +  iiXc' (n)e‘ ,  (8) 

~WcT(n  +  \)Xc(n  +  \).  (9) 


_(n+l)2 

Multiply  (9)  with  Q  ^”+l?vF  2  ,  and  define  en  =  Q  nvF  2  ecn , 

n  —  1,2,... ,  which  is  the  unchirped  version  of  the  predictor  error 
signal  for  chirped  process  using  LMS,  (9)  becomes, 

entX=x^-WT{n  +  \)X(n+\)  (10) 

where  W(n  +  \)  is  the  corresponding  predictor  weight  in  the 
transformed  domain.  (8)  can  be  shown  to  become 

W{n  +  l)  =  VA'[W(n)  +  fiX(n)em).  (11) 

where 

V&  =V4  -,*diag{'V'.Y,-,'VM}  (12) 

is  the  chirp  rotation  matrix.  Note  that  alter  unchirping,  the  signal 
is  a  stationary  real  baseband  signal.  Since  en  is  the  unchirped 

2 

version  of  en ,  E[|p„|  1=E[|Q”4/  2  ecn  \2]  =  E[\eLn  \2] ,  i.e., 

the  MSE  of  LMS  algorithm  on  chirped  process  xcn  equals  the 
MSE  of  a  different  I  MS  algorithm  on  a  corresponding  stationary 
process  Xn  with  a  LMS  predictor  of  the  same  length  M  and  step- 
size  fl  .  (10)  and  (11)  will  be  referred  to  as  rotated  L  MS 
algorithm,  since  the  only  difference  of  these  equations  with  the 
standard  L,MS  algorithm  for  stationary  input  signals  (4)  (5)  is  that 

the  weight  vector  is  rotated  by  chirp  matrix  VA  after  each 
update.  The  rotation  of  updated  weight  will  give  extra  MSE  in 
excess  of  the  normal  MSE  of  the  standard  LMS  algorithm  on 
stationary  signals. 

3.2  Error  Transfer  Function  Approach  for  Rotated  LMS 
Predictor 

First,  we  decompose  the  L,MS  predictor  weights  into  a  sum  of 
time-invariant  Wiener  predictor  and  a  time-varying 
misadjustment  component 

fV(n)  =  fV0  +  Wmix(n).  (13) 

Since  there  is  a  directional  rotation  after  each  weight  update  in 
(II),  is  not  zero  mean,  it  can  be  further  decomposed  as 

K,,  (»)  =  (n)  +  ivm,  (n),  (14) 

where  Wmls(n)  =  E\WmU(rij\  is  the  mean  weight  misadjustment 

corresponding  to  the  weight  fluctuation  caused  by  weight 
rotation.  From  Equation  (11),  the  mean  weight  misadjsutment  is 
given  by 

(n+\)  =  Vl(l-nRx)Wmu(ri)-(l-  Vl  Wn  (15) 

when  i.e.,  the  adaptive  filter  reaches  steady  state, 

wms=-(\-nv,Rxy'\w0  (i6) 

where  A  =  VA  —  J  . 

This  steady-state  mean  weight  misadjustment  term  is  equivalent 
to  the  lag  weight  misadjustment  of  chirped  input  process  using 
LMS  algorithm  as  shown  in  [5]. 


(26) 


The  recursive  weight  update  equation  (11)  can  now  be  written  as 

W(n)  =  Vl"W(  0)  +  fiJjV?"-J)XU)eJ  (17) 

M 

The  error  process  en  can  be  shown  to  satisfy  the  recursive 
difference  equation 


=C, 2. 

with  MSE  given  by  JtM  =  £[|  en  |J ] .  The  optimal  MSE 
estimator  using  the  same  information  is  given  by 

2...,x_]  =  £[xJx„  (27) 


e„+H^ejXHUWlin'J)X(»)  =  x„  -\W0  +  WmJX{n)  (18) 
j‘\ 


using  the  approximation  XT (j)X(n)  «  Mrx(n  —  j) ,  where 
rx(k)  is  the  autocorrelation  of  input  signal  vectors,  and 

XH(j)V^-j)X(n)  «  Mrxc(n-j)  yf  «  1  (19) 

Equation  (18)  is  approximated  by  a  standard  difference  equation 
with  constant  coefficients  as 


en  +  PM  X  rx(n~  J'K  “  ^ f  X(n)  (20) 


We  can  interpret  the  steady-state  LMS  prediction  error  en  as  the 
output  of  a  time-invariant  linear  system  with  transfer  function 
H(z)  given  by 


where 


H(z)  = 


1 

]  +  liMR(z) 


m- 1 


(21) 


(22) 


driven  by  the  wide-sense  stationary  error  process 
xn  —  [W0  +  Wms  f  X(n) .  The  MSE  of  the  LMS  predictor  is 

J*  |l~»'oC')-^(.’)|2 S„(r)*  (23) 

A/+A-1  _  M+A  1  _ 

Where,  Wn(z)=  £  W"(]  (  j)X‘  ,  W^(z)=  £ 

j- a  y=A 

arc  the  transfer  functions  of  Wiener  predictor  and  mean  weight 
mis  adjustment  of  rotated  LMS  predictor  respectively.  S^z)  is 
the  PSD  of  the  stationary  input  process. 

The  error  transfer  function  approach  can  also  be  applied  to  the 
Normalized  LMS  (NL  MS)  algorithm  as  defined  below  [3] 

fV(n  +  \)  =  fV(n)  + - V-^rX\n)e„  (24) 

||  A" (m)  ||2  " 

with 


\  +  HR{z)l{P,+Pn) 

4.  HOUND  OF  MULTIPLE-STEP  PREDICTOR 

Using  the  recursive  equations  (4)  (5),  it  follows  from  [2]  that  the 
L  MS  estimator  is  a  nonlinear  function  of  the  input  signal  and  can 
be  written  as 


Thus  the  optimal  estimator  for  A- step  predictor  is  the  1-step 
infinite-length  Wiener  predictor. 

The  reason  that  the  multiple-step  predictor  can  outperform  fixed 
Wiener  filter  is  that  the  additional  information  which  maybe 
available  to  adaptive  filter  but  not  available  to  fixed  Wiener  filter 
has  two  parts,  ^  ,  JC„_2 , x„_(A  J  and 

» ■  The  main  contribution  is  the  first 
part  since  for  narrowband  signal,  the  correlation  of  the  first  part 
with  current  signal  X„is  larger  that  the  correlation  with  the 

second  one.  Note  also  the  correlation  of  Xn  with  the  first  part  is 

also  larger  than  the  correlation  of  Xn  with  input  signal 

. . X„  (4+M_i,}  no  matter  where  the  signal  pole  is 

located.  With  the  increase  of  predictor  bulk  delay  A ,  there  is 
more  information  available  to  adaptive  filters  than  to  the  finite 
Wiener  filters. 

5.  COMPARISON  OF  LMS  AND  RLS  ALGORITHMS 

The  weight  update  equation  of  exponentially  weighted  RLS 
algorithm  is 

tV(n+  1)  =  W (n)  +  O  \n)X‘ (n)e„  (28) 

n 

where  d>(>?) =  5^ Xn~‘X{n)XH  (n)  is  the  input  signal 

/*o 

autocorrelation  estimate  at  time  n ,  and  A  is  the  forgetting 
factor  of  RLS  algorithm.  Using  the  same  approach  as  in  the  LMS 
algorithm,  the  steady  state  error  update  equation  of  RL,S 
algorithm  is  given  by 

0)0  \j)X(n)  =  d„-WjX(n)  (29) 

y-o 

At  steady  state,  using  the  following  approximations, 

O'OXl-A)/?-1  (30) 

XH  w  ' U)X(n)  -  (1  - X)irace(R-'E[X(n)X"  0)])  (31) 
define  cr(j)  =  (1  - X)trace(R  '  E[X*  (n)XT  (n  - y)]) ,  The  left 
hand  side  of  (29)  is 

+  =x„-lV0rX(n)  (32) 

)•  o 

For  simplicity,  we  only  compare  the  two  algorithms  with 
stationary  input  signals.  The  error  feedback  equation  for  LMS 
algorithm  (20)  becomes 

+  X  cljeJ  =x-~  <  X  («)  (33) 


where  clj=fJ.Mrx(j).  Figure  2  is  a  plot  of  cl}  and  crjt 
7  =  1, 2,..., 50  for  a  narrowband  AR1  input  signal  embedded  in 

AGWN,  with  AR1  pole  location  at  a  =  0.99,  snr  =  \0dB , 
adaptive  Filter  length  M  =  25 .  Note  that  the  error  feedback 
coefficients  curve  of  the  RLS  predictor  has  a  null  for  small  j  , 
which  means  that  the  contributions  from  the  most  recent 
prediction  errors  to  the  current  estimate  at  time  index  n  are 
nulled  out.  For  the  LMS  predictor  the  most  recent  prediction 
errors  contribute  more  to  current  estimation  than  the  time  delayed 
prediction  errors. 

6.  SIMULATIONS 

For  a  chirped  AR1  process  using  the  LMS  predictor 

r/0)  =  y  2  aJ  (34) 

The  feedback  transfer  function  is  thus 

,  (35) 

I  -gj 

(A|  Af-lj 

where  ac  =  (fY  2  ,  gc  =  (1  —flMPs  )ac . 

The  MSE  of  the  LMS  predictor  can  be  computed  from  (23)  using 
(35). 

Figure  3  is  a  plot  of  MSEs  under  three  conditions:  using  transfer 
function  approach,  the  independence  assumption  and  simulation 
results  for  a  chirped  input  signal  with  chirp  rate  (p  =  5ne-5 , 

signal  pole  location  a  =  0.999 ,  signal  power  Ps  =  1 , 

SNR=0dB,  Filter  length  M=2.  IEese  results  arc  compared  to  the 
MSEs  obtained  for  a  Finite  length  Wiener  predictor  and  the 
optimal  estimator.  It  can  be  seen  that  in  a  small  range  of  adaptive 
Filter  stepsize  parameter  fd  ,  the  MSE  using  error  transfer 
function  approach  is  smaller  than  the  MSE  of  Finite  Wiener 
predictor.  Simulations  and  transfer  function  approach  show  that 
the  nonlinear  effect  is  only  observable  for  very  small  filter 
length,  very  narrow  bandwidth  input,  and  a  range  of  input  signal 
SNR,  since  it  is  only  under  this  condition,  the  information 
{x*-lM+\)xn  (M+2)  •••>*—}  ^at  *s  available  to  adaptive  filter  but 

not  available  to  Fixed  Wiener  Filter  can  have  effective 
contributions  to  the  prediction  of  current  signal. 

Figure  4  plots  the  MSEs  for  40-step  NLMS  and  RLS  predictors 
for  a  stationary  and  a  chirped  input  signal  with  chirp  rate 
(p  =  $jze  —  4  ,  signal  pole  location  a  =  0.99  ,  input  signal  power 

P  =  1 ,  SNR=20dB  and  the  length  of  the  predictor  is  M=25.  For 
NLMS  predictors,  the  MSEs  obtained  by  transfer  function 
approach  and  simulation  results  are  given  for  both  stationary  and 
chirped  inputs.  The  simulation  results  of  RLS  predictor  for 
stationary  input  are  also  plotted  to  show  that  the  nonlinear  effects 
arc  negligible  for  RLS  algorithms.  Comparing  the  results  with 
Figure  3,  the  range  of  adaptive  Filter  parameter  /A  over  which  the 

LMS  predictors  outperform  fixed  Wiener  predictor  is  much 
larger,  and  the  magnitude  of  the  nonlinear  effect  is  significant  at 
optimal  stepsize.  The  reason  for  this  is  that  for  multiple-step 
predictors,  additional  information  which  is  available  to  adaptive 
predictors  but  not  available  to  Fixed  Wiener  Filter  has  two  parts, 


2  >,,,»-*rt-(A-l)  i  ant^  {■^rt-(A+A/)’^n-(A+M+l)»,,,»-^— }  ’ 

for  1-step  predictors,  only  the  second  part  is  available.  The  main 
contribution  to  nonlinear  effects  is  the  First  part  since  with  the 
increase  of  time  delays,  the  correlation  between  the  desire 
response  xn  and  the  additional  part  becomes  smaller,  thus  has 
less  contribution  to  current  estimation. 

Figure  5  plots  of  MSEs  of  fixed  Wiener  predictor,  optimal 
estimator  and  MSE  of  simulations  achieved  at  optimal  stepsize 

/X{)pt  as  a  function  of  delay  step  A  .  It  shows  that  with  the  above 
parameters,  the  LMS  adaptive  filter  start  to  outperforms  Wiener 
Filter  for  A  >  5  and  with  the  increase  of  delay  step,  the 
nonlinear  effect  become  more  significant 

Figure  6  is  a  plot  of  MSEs  versus  input  signal  pole  location  a 
for  a  40-step  predictor.  It  shows  that  the  range  of  input  signal 
pole  location  over  which  nonlinear  effect  is  observable  is  from 
about  0.75  to  around  1 .  This  range  is  also  much  larger  compared 
to  the  1  -step  predictor  case. 

7.  CONCLUSIONS 

In  conclusion,  this  paper  shows  that  it  is  possible  for  an  adaptive 
transversal  prediction  Filter  to  outperform  the  Wiener  predictor  of 
the  same  length  for  narrowband  chirped  input  signal  embedded  in 
Added  White  Gaussian  Noise  (AWGN).  These  cases  arise  when 
the  adaptive  filter  uses  more  information  than  the  fixed  Wiener 
filter.  It  shows  that  the  nonlinear  effect  is  observable  for  1-step 
predictors  for  a  narrow  range  of  input  signal  and  adaptive  filter 
parameters,  and  it  is  significant  for  multiple-step  predictor  for  a 
wide  range  of  parameters.  A  chirp  transform  is  defined  to  convert 
the  chirped  input  signal  to  real  baseband  stationary  input  signal, 
and  a  transfer  function  approach  is  derived  for  chirped  input 
signals  to  compute  the  total  MSE  of  adaptive  LMS  predictors. 
The  performance  of  I -step  infinite-length  Wiener  predictor  is 
used  as  the  optimal  estimation  to  bound  the  performance  of 
adaptive  A  —step  predictors.  The  nonlinear  effects  are  much 

larger  for  the  LMS  predictor  than  the  RLS  predictor  since  LMS 
uses  information  from  the  past  errors  more  effectively  than  the 
RLS  predictor  does. 
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Figure  1.  LMS/RLS  A  — Step  Predictor  Structure 


Figure  4.  MSEs  of  NL  MS  and  RLS  40-stcp  predictors  as  a 
function  of  adaptation  constant,  with  SNR  =  20dB,  M=25,  a  = 
0.99,  chirp  rate  y/  =  Site  —  4 
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Figure  2.  Error  feedback  coefficients  of  L,MS  and  RLS  predictor 
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Figure  3.  Comparison  of  MSEs  of  1-stcp  LMS  predictor  for  veiy 
narrowband  input  signal  as  a  function  of  adaptation  constant  fl  , 

with  a=0.999,  M  =  2,  SNR=1,  chirp  rate  y/  =  5Ke  -  5  . 


Figure  5.  MSEs  of  NLMS  predictor  at  optimal  stepsize  as  a 
function  of  prediction  delay  A ,  with  SNR  =  20dB,  M=25,  a  = 
0.99,  chirp  rate  y/  =  57te  —  4 
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Figure  6.  MSEs  of  40-stcp  NLMS  predictor  at  optimal  stepsize  as 
a  function  of  input  signal  pole  location  a,  with  with  SNR  =  20dB, 
M=25,  chirp  rate  y/  =  5 Ke  -  4 


